Dependence Driven Execution for Data Parallelism

This paper proposes an efficient run-time system to schedule general nested loops on multiprocessors. The work extends existing one-dimensional loop scheduling strategies such as static scheduling, affinity scheduling and various dynamic scheduling methods. The extensions are twofold. First, multiple independent loops as found in different branches of parbegin/parend constructs can be scheduled simultaneously. Secondly, multidimensional loops with dependencies and conditions can be aggressively scheduled. The ability to schedule multidimensional loops with dependencies is made possible by providing a dependence vector as an input to the scheduler. Based on this application-specific input, a continuation-passing run-time system using non-blocking threads efficiently orchestrates the parallelism on shared memory MIMD and DSM multicomputers. The run-time system uses a dependence-driven execution which is similar to data-driven and message-driven executions in that it is asynchronous. This asynchrony allows a high degree of parallelism.